Many sophisticated GUI's (Graphical User Interface) and display systems have attempted to cope with the problem of making contextually related and sequentially or hierarchically organised data and information ‘findable’ in a visually intuitive, meaningful and user accessible way particularly when displaying many grouped visual digital data elements representing categories, directories, folders, files and the navigational or functional elements of a website, application or operating system. Traditional displays however, must severely limit the number of data elements that can be displayed on a screen without visually confusing the user. In this patent the visual representations of such data elements to be displayed to a user on a screen, including but not limited to those just listed, will be referred to as ‘icons’. The use of the words ‘screen’ or ‘display’ will mean any device that actively or dynamically presents digital data visually to a user and with which the user may interact using typically a pointing device such as a mouse, touch sensitive screen or other user interactive device.
From the users perspective icons can represent, often in a simple stylised way, individual or grouped multimedia items such as video and audio items, photo's, retail items such as books or hotels, personal information items such as contacts etc. They may also represent user interface ‘windows’, website links (URLs), process control elements, electronic program guide (EPG) data or video game elements or indeed any individual data element that a user may need to view or interact with on a screen. In all these cases the icons are visual links to static or functional (control) data elements residing within or connected to the storage and processing elements of the equipment being used. Such icons may represent nodes in a data tree from which a user can select a series of static or active elements within different levels of the tree to locate a particular element or functional goal. This search may be done in a structured, goal oriented way or in a more random ‘browsing’ fashion.
A typical presentation of such icons to a user could be the arrangement on a screen of the results of a user database query. Since the beginnings of such screen data presentations, one of the principle methods to display results has been in a manner resembling a printed paper catalogue: Multiple icons or graphic data elements are laid out on the users screen as one or more consecutive ‘pages’ or screens populated with a static ‘flat’ 2D linear array consisting of rows and columns of icons containing images, symbols and/or text frequently with additional descriptive or title text adjacent to the icons. With such presentations a user may normally view only one page or screen at a time. FIG. 1a, indicates the flat 2D linear array approach.
More recently methods have been revealed in which icons not only move around the x-y axes of a flat screen but are dynamically resized as if moving in the third ‘z’ dimension perpendicular to the screen surface. Icons in these displays often appear to follow normal rules of perspective and can appear to move in front of and obscure each other. Typical of this type of icon presentation is the rotating 3d Carousel device which has appeared in many forms and suffers from similar problems common to the 2d arrangements described above: with more than just a few icons in a carousel, only those icons at the ‘front’ of the device are clearly visible—those at the sides and towards the rear are reduced in size and stacked on top of each other concealing some or all of their contents and any contextual boundaries or relationships between adjacent and non adjacent icons. FIG. 1c demonstrates this type of carousel arrangement.
Even in those dynamic 3d systems that provide a facility to reveal individual title or descriptive information about each icon as it is ‘moused’ over by the users cursor, the same failing is apparent: It is extremely difficult in a reasonable time to establish the contextual boundaries and relationships between adjacent and non adjacent icons and groups of related icons in such a display and thus visually locate areas of the presentation where icons of relevance to the user may be found. This creates a substantial load on the users cognitive and memorising abilities. User success with such systems is often a matter of chance. Anecdotal evidence and data derived from usability studies suggest users can feel overwhelmed or suffer from ‘cognitive overload’. This failing of such prior art can often result in wasted time, frustration and users giving up before fulfillment is reached. In a commercial environment this can represent lost sales and in more critical applications the consequences can obviously have a far greater importance.
Recent dynamic 3D (i.e. having visual depth cues such as apparent or ‘virtual’ perspective, depth and motion) displays employing such moving ‘streams’ or ‘trains’ of multiple icons have enabled users to browse through a continuous linear or a rotating ‘Carousel’ presentation display in which the arrangement of icons can move under user or system control. In such presentations the icon streams are either looped or non-looped: Looped icon displays include continuous and circular ‘carousels’ or linear streams of icons with provision for those icons that stream off one side of the screen to reappear from the other side of the screen. Non-Looped displays do not provide for icons to reappear—the user must move the icon stream in the reverse direction to review icons. Two of the most prominent examples to date of 3D icon presentations are the ‘linear’ style presentations of iTunes ‘Coverflow’ (Apple Inc.) (see FIG. 1d) and the open source ‘Cooliris’ (FIG. 1e) which have appeared recently on websites and other applications. Both display methods employ linear, non looped user controlled single or multilayered arrays, streams or ‘walls’ of icons, often numbering in the hundreds, arranged either like a spread out ‘deck of cards’ flowing from one side of the screen to the other under user control or a multilayered flat wall of images streaming towards and past the user. The ‘Coverflow’ style ‘deck of cards’ arrangement has a central position in the stream where icons enlarge and spacing between icons increases causing the ‘deck’ to appear to open out thus allowing each icon in turn to be presented to the user in a ‘front facing’ position, fully revealing its contents. In this way the user can examine in a serial manner, each icon in the display. With such systems the user generally controls icon movement by means of a ‘mouse’ or other pointing device employing a clicking and dragging method or more recently with a touch screen. With the Coverflow arrangement, unless the number of icons is small, icon visibility to either side of the central position become more and more restricted as icons are stacked up on each other increasingly concealing the contents of those behind them. With a small number of icons it is possible to give each icon a title or very short description but as the number of icons increases the display of descriptive or title text is usually restricted to a single front position only. Alternatively a small ‘pop-up’ text window (sometimes known as a ‘tool-tip) appears as the user's cursor is positioned over individual icons (a technique known as ‘mousing’ over). With the flat moving array arrangement typified by ‘Cooliris’, the icon array moves as a multi-level wall of icons under the users control.
Findability—The primary problem for users with all such currently disclosed flat, 2D or 3D linear or looped multi-icon presentations is a lack of visibility of the icons, particularly when there are a large numbers of them overlaying each other as demonstrated in FIGS. 1b, 1c and 1d. More particularly, the lack of visible contextual boundaries between individual icons or groups of icons within the whole array gives the user no clues as to where to look for specific icons or groups of icons as seen in FIG. 1b. This often critical factor we will refer to as the ‘findability’ problem. A specific example illustrates the point: A user issues a query or search on a system containing a catalogue of classical music items. The search is for Chamber Music string Quartet recordings. Typically the results might be presented to the user as several pages of icons, one page at a time, representing such works by all composers in the catalogue (FIG. 1b indicates the problem). The first issue for the user is how has the system sorted them? Is it alphabetically by composer or is it by title or by some other criteria? Even where the system does use an explicit sorting criteria, the boundaries between the sort criteria elements are almost always not visible or they may be on pages not visible to the user, as can be seen in FIG. 1b. How would the user locate the Chamber Music string quartets of Mozart, or Beethoven? And where are those pages within the whole set of results pages presented to the user. To sum up: The user is unlikely to get any idea of how many icons belong to which composer, how are the results (icons) organised and where are the boundaries between each subset or sub group (in this example ‘composers’) or on which page they are located. In the case of many consecutive pages of icons in a 2d array, users have little or no idea where to look for relevant results—The user can normally only see one page or screen of icons at any time and it may not be obvious on what basis non adjacent and even adjacent icons have been contextually arranged. They may have to look through all pages to find icons relevant to their task or goal. More generally, icons that are hierarchically organised may be sorted and arranged numerically or alphabetically by title, author, shoe size, colour etc or some other criteria appropriate to the application. However, since only a small subset is visible to the user at any one time it will not be obvious what the relationships are or where the hierarchical boundaries or breaks occur in the other unseen pages of the presentation. Consequently, locating or finding pages with relevant results can be just a matter of chance because the boundaries between different contextual subsets or nodes within the whole group are unmarked and hidden amongst the unseen screen ‘pages’ or stacked icons in an often irregular and unpredictable manner.
Other 3D devices within the prior art and of even greater sophistication include a parent and child carousel arrangement where each icon in a carousel represents a node in a complex data structure. Selecting an icon (or tree node) in a ‘Parent’ carousel initiates the creation of a child carousel. The child carousel can then produce even more offspring. This kind of device is typified by U.S. patent application Ser. No. 12/109,530. Whilst having the ability to display a multi level data tree, this device cannot reveal at a glance the contextual boundaries and relationships amongst the icons displayed in any particular carousel for the reasons discussed earlier. (See FIG. 1g)
To sum up, all such multi-icon systems suffer from a single major failing of not indicating to the user at a glance the relationships between adjacent and non adjacent icons and groups of icons within the total of those presented on screen or the likely location of icons that will be of interest to the user. The user may have to visit all icons presented to find a single icon of relevance.
Examples of prior art:                A ‘Coverflow’ clone can be seen at http://www.weberdesignlabs.com/blog_files/itunes/iTunesAlbumArt.html (Coverflow is a trademark of Apple Inc.)        Apple Inc. Coverflow can be seen by downloading Apple iTunes        Another Coverflow clone http://www.yofla.com/flash/cover-flow/        ‘Cooliris’ can be seen at http://www.cooliris.com/        
Existing Patents and applications include:                U.S. Pat. No. 6,335,737 B1 Video Display on a graphical Interface (Jan. 1, 2002)        US 2008295037A1 Method and Apparatus for Generating & 3D Carousel Tree Data Visualisation and Related Device (Nov. 27, 2008 U.S. patent application Ser. No. 12/109,530)        